Interactive performance direction for a simultaneous multi-tone instrument

ABSTRACT

A musical instrument performance solution is described. Labels with visual indicators provide a reference to performers such that a proper combination of instrument inputs may be selected at the appropriate time. The visual indicators include colors and/or shapes. The visual indicators may be presented using differently-colored lyrical text, where each color corresponds to a set of notes. Each set of notes may for a chordal group such as a triad. The visual indicators may be associated with labels that are able to be adhered to various instrument inputs such as keys of a keyboard or piano.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent Application Ser. No.14/798,317, filed on Jul. 13, 2015. U.S. patent application Ser. No.14/798,317 claims priority to U.S. Provisional Patent Application Ser.No. 62/025,276, filed on Jul. 16, 2014.

BACKGROUND

Various ways to learn to play an instrument are available. For instance,many users may receive person instruction, utilize playbooks or otherstudy materials, review video or audio lessons, etc.

Such methods are time-consuming, expensive and boring. Existingsolutions require users to master concepts and theory and/or includepractice materials that are not current or interesting to the student.

Thus there is a need for a solution where a user is able to quickly,cheaply, and easily play a variety of materials without having to mastertheory, practice exercises, or carry out other non-performance relatedendeavors. In addition, such a solution should eliminate the need forfluency in any particular language.

SUMMARY

Some embodiments provide ways to play simultaneous multi-toneinstruments without requiring hours of practice or any understanding ofmusic theory. Some embodiments provide a labelling kit. Such a labellingkit may be able to be applied to various appropriate features of aninstrument (e.g., keys of a piano or other keyboard, frets of a guitar,etc.). The labelling kit may include coordinated elements (e.g.,multiple elements may share a common color, arrangement, etc.).

Some embodiments may provide accompanying media content that has beenoptimized for use with the labelling elements. Such content may include,for instance, printed content, content displayed via a web resource,content displayed via an application or app, etc.

The content may be retrieved from various sources and optimized for usewith the labelling elements. The content may be optimized in variousappropriate ways, where such optimization may depend on the source ortype of content. Optimized content may be presented to a user withvarious cues (e.g., visual cues, audio cues, etc.) that correspond toelements of the labelling kit. For instance, cues may include displayedcolors and/or shapes that correspond to various label elements. A usermay interpret the cues by selecting the instrument elements or locationsthat correspond to the appropriate labels.

Users may be able to select from among various performance levels (e.g.,lesson, beginner, intermediate, expert, etc.). Such selection may atleast partially direct the information provided to the user. Forinstance, a beginner (or user that selects a particular lesson or lessontype) may be presented with only a bass line (or melody, or chordprogression) of a song, rather than all elements.

Some embodiments may provide optimized content via a graphical userinterface (GUI). Such a GUI may be provided via an appropriate userdevice (e.g., a personal computer or “PC”, tablet, smartphone,television or “TV”, etc.). Optimized content may be distributed byvarious network-connected resources (and/or other appropriateresources).

The preceding Summary is intended to serve as a brief introduction tovarious features of some exemplary embodiments. Other embodiments may beimplemented in other specific forms without departing from the spirit ofthe disclosure.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The novel features of the disclosure are set forth in the appendedclaims. However, for purpose of explanation, several embodiments areillustrated in the following drawings.

FIG. 1 illustrates a facing view of a labelling kit of some embodiments;

FIG. 2 illustrates a top view of a labelling kit of some embodimentsafter application to a keyboard;

FIG. 3 illustrates a schematic block diagram of an interactive system ofsome embodiments;

FIG. 4 illustrates a graphical user interface (GUI) used by someembodiments during an interactive performance session;

FIG. 5 illustrates a flow chart of an exemplary process used by someembodiments to provide interactive content for playback; and

FIG. 6 illustrates a schematic block diagram of an exemplary computersystem used to implement some embodiments.

DETAILED DESCRIPTION

The following detailed description describes currently contemplatedmodes of carrying out exemplary embodiments. The description is not tobe taken in a limiting sense, but is made merely for the purpose ofillustrating the general principles of some embodiments, as the scope ofthe disclosure is best defined by the appended claims.

Various inventive features are described below that can each be usedindependently of one another or in combination with other features.Broadly, some embodiments generally provide ways to perform using amulti-tone instrument. Some embodiments may include various labellingfeatures that may each be associated with various elements or locationsof an instrument (e.g., keys of a piano). In addition, some embodimentsmay provide content that includes various cues associated with thelabelling features such that a user is directed as to which elements ofthe instrument should be manipulated at which time.

Several more detailed embodiments are described in the sections below.Section I provides a description of a labelling kit provided by someembodiments. Section II then describes a system architecture of someembodiments. Next, Section III describes a user interface provided bysome embodiments. Section IV then describes methods of operation used bysome embodiments. Next, Section V describes various alternative usagescenarios of some embodiments. Lastly, Section VI describes a computersystem which implements some embodiments.

I. Labelling Features

FIG. 1 illustrates a facing view of a labelling kit 100 of someembodiments. In this example, and various examples and description thatfollow, a piano or other keyboard instrument may be used. One ofordinary skill in the art will recognize that various elements of thedisclosure may be implemented with various other types of simultaneousmulti-tone instruments (e.g., strings, percussion, etc.). Furthermore,some embodiments may be utilized with single-tone instruments (e.g.,brass, woodwinds, etc.) in a similar manner to that described inreference to simultaneous multi-tone instruments. In addition, differentsets of labels may be provided for different types of music, differentkey structures, etc.

In this example, the labelling kit includes a first set of labelelements 110, a second set of label elements 120, and a third set oflabel elements 130. Different embodiments may include different numbersand/types of sets of elements.

The first set of label elements 110 may include a number of elements 140including various visual cues 150. The first set of label elements 110may be applied to a first set of keys of a piano (and/or other inputelements associated with other instruments). In this example, the firstset of label elements may be associated with keys played using the lefthand (also referred to herein as “bass” keys).

The visual cues 150 are represented as ovals with different fillpatterns. Various different shapes (e.g., round, square, polygonal,irregular, etc.) may be used. Such cues may have various patterns orother identifiers (e.g., color, texture, etc.). Each label element 140may be sized appropriately for the corresponding instrument element(e.g., the “white” keys corresponding to the natural notes of a C majorscale).

The second set of label elements 120 may include elements 160 withvarious visual cues 170. The second set of label elements 120 may beapplied to a second set of keys of a piano (and/or other input elementsassociated with other instruments). In this example, the second set oflabel elements 120 may be associated with keys played using the righthand (also referred to herein as “harmony” keys).

The visual cues 170 are represented as rectangles with different fillpatterns. The fill patterns may correspond to the fill patternsassociated with cues 150. As above, the cues 170 may have variouspatterns or identifiers. Each label element 160 may be sizedappropriately for the corresponding instrument element (e.g., the“white” keys corresponding to the notes of a C major scale).

The third set of label elements 130 may include elements 180 withvarious visual cues 190. The third set of label elements 130 may beapplied to a third set of keys of a piano (and/or other input elementsassociated with other instruments). The third set of label elements inthis example may also be associated with keys played using the righthand.

The visual cues 180 are represented as rectangles with different fillpatterns. The fill patterns may correspond to the fill patternsassociated with cues 150 and 170. As above, the cues 180 may havevarious patterns or identifiers. Each label element 190 may be sizedappropriately for the corresponding instrument element (e.g., the“black” keys corresponding to the notes not included in a C majorscale).

The various label elements 140, 160, and 180 may include exposedsurfaces that are laminated and/or otherwise protected against wear whena user manipulates an instrument control element associated with alabel. In addition, each label element may include various adhesivesthat may allow the element to adhere to the instrument control element.As another example, each label element may include materials thatautomatically adhere or “cling” to a surface associated with theinstrument (e.g., cling form labels may be able to adhere to ivory,plastic, and/or other typical keyboard materials without affecting thefinish of the keys). Each label may be adhered such that the label doesnot slide or peel during use. Some embodiments may include labels withdouble-sided laminates that help prevent bleed into the keys and fadingfrom use.

The kit 100 of FIG. 1 is presented for exemplary purposes. Such a kitmay be implemented in various different ways without departing from thescope of the disclosure. For instance, different numbers of elements,different shapes or colors of cues, etc. may be used. As anotherexample, different instruments may be associated with different kits.

FIG. 2 illustrates a top view of a labelling kit 200 of some embodimentsafter application to a keyboard 210. Such a kit 200 may be similar tokit 100 described above.

In this example, the third set of label elements 130 has a blackbackground to correspond to the black keys of the keyboard 210.Different embodiments may include labels of different shape, size,background, etc.

In this example, placement of each element is indicated by the dashedarrows. The section of keyboard 210 may begin at lower C (i.e., the Cbelow middle C) and extend to the second D above middle C. The first setof labels 110 may be applied sequentially beginning at lower C and thesecond set of labels 120 may be applied sequentially beginning at middleC. The third set of labels 130 may be applied as shown. In this example,the “base key” is C major (or A minor).

Different embodiments may include different groups and/or configurationsof labels that may be applied in various different ways than shown. Inaddition, some embodiments may include multiple iterations of each labelset such that, for instance, multiple additional intervals may be ableto be placed along the keyboard.

The labels may be configured such that each color (or fill, texture,etc.) is associated with a note or set of notes. In this example, thefirst set of elements 110 may each be associated with a single note,starting from lower C (which may generally define the base key orperformance key of the tune). The second set of elements 120 may includelabels associated with various sets of notes. For instance, the sets ofnotes may be arranged to define major triads, minor triads, etc. Thesets of notes may correspond, such that a color from the first set ofelements 110 and a color from the second set of elements 120 definerelated notes (or “chords”). The third set of elements 130 may likewisebe associated with various note combinations or chords that may bespecified using a single color (and/or other reference).

In some embodiments, multiple colors may be associated. For instance, ayellow bass note may be associated with dark yellow and light yellowharmony notes, where the dark and light yellow may represent chordvariations that use the same bass note (e.g., a major triad and a minortriad).

II. System Architecture

FIG. 3 illustrates a schematic block diagram of an interactive system300 of some embodiments. Such a system may be used in conjunction with akit such as kit 100. The system may include one or more user devices310, a server 320, a storage 330, and/or a network 340.

Each user device 310 may be a device such as a tablet, smartphone, PC,etc. The user device may include a display and/or various interfaceelements (e.g., touch screen inputs, buttons, keypads, audio inputs,etc.). Each user device may be able to communicate across network 340 toretrieve content from server 320.

The server may be able to store and retrieve content from storage 330.The server 320 may be able to communicate with multiple user devices 310across network 340. The storage 330 may be able to store content,instructions, data, etc. The storage may be associated with an externalsystem and be accessed via an appropriate interface (e.g., anapplication programming interface or “API”).

The network 340 may include various wired and/or wireless networks. Forinstance, the network may include one or more local-area networks (e.g.,Ethernet, WiFi, etc.), one or more extended-area networks (e.g.,cellular networks, the Internet, etc.), and/or various othercommunication pathways (e.g., wired connections, transferable storages,etc.).

During operation, various users may access the system via user devices310 across network 340. The server 320 may provide and/or generatecontent optimized for use with the labelling kit of some embodiments.

III. User Interface

FIG. 4 illustrates a graphical user interface (GUI) 400 used by someembodiments during an interactive performance session. Such a GUI may beprovided by a device such as user device 310. In this example, the userdevice may be a tablet with a touch screen display 410. GUI 400 isprovided for example purposes only and the specified notes do notcorrespond to any actual tune.

As shown, the display may include lyrics and/or other elementsassociated with some media content. In addition, this example includesindicators 420 and 430 that may be used to show which labelledinstrument elements to activate at each point within the media content.Alternatively, some embodiments may include colored text thatcorresponds to the various label colors. Thus, as the color of the lyrictext changes, the user may play corresponding notes on the keyboard.Some embodiments may use a single color to define all notes (bass andharmony), while other embodiments may allow for multiple colors to bespecified (e.g., indicator 420 may be used to indicate a bass note,while indicator 430 indicates a set of harmony notes).

Different indicators (e.g., differently shaped, colored, and/orotherwise differentiated) indicators may be used to designate theappropriate label elements to activate. The indicators may be displayedand/or spaced at various appropriate intervals to indicate executiontime.

Different embodiments may present the directions in various differentways (e.g., by providing cues along a timeline, by providing audio cues,by including a moving indicator showing a position along the lyrics,etc.).

In addition, a user may be able to control the presentation in variousappropriate ways (e.g., by pausing or stopping playback, by slowingplayback, etc.). A user may be able to select a portion of the mediacontent for repeated playback and practice.

Some embodiments may provide links to additional resources, such asexample use videos, related content items, etc. For instance, oneelement may link to an original video and/or streaming music resourceassociated with a song. Such a resource may allow a user to call up theoriginal song as a guide during practice or performance and use theoriginal song to strengthen ear training, observer technique, etc.

As another example, a different element may link to an instructionalvideo. Such videos may offer short cuts to create a rhythm, arpeggiationor other pronunciation using the labelling system of some embodiments.In this way, users may be able to identify the needed keys for aparticular song, indirectly memorize commonly used keys, and hasten thepiano experience.

IV. Methods of Operation

FIG. 5 illustrates a flow chart of an exemplary process 500 used by someembodiments to provide interactive content for playback. Such a processmay be executed by a device such as user device 310 which may work inconjunction with server 320 and/or other system elements. The processmay begin, for instance, when a user accesses a web resource of someembodiments, launches an app, etc.

When a user first accesses the system resource, some embodiments mayregister and subscribe the user. The user may then be able to access acatalog of songs (e.g., songs placed in remote storage 330).

As shown, process 500 may identify (at 510) a user and/or a set ofattributes. The user may be associated with a username and password, forinstance. The attributes may include instrument type, user performancelevel, biographical information, etc. In addition, the user attributesmay include information such as preferred styles, artists, etc.

Next, the process may establish (at 520) a base key. Such a base key maybe selected based on various relevant criteria, including, for instance,instrument type, user preference, style, etc. The process may thenreceive (at 530) a selection of some media item (e.g., a song, tune,etc.). Such a selection may be received using various appropriate GUIelements (e.g., displayed lists, menus, etc.).

The process may then determine (at 540) whether the media item haspreviously been processed using appropriate attributes for the currentuser (e.g., instrument type, user level, etc.). In some embodiments,only previously-processed items may be provided for selection. In suchcases, content may be identified and processed based on various relevantcriteria.

If the process determines (at 540) that the media item has not beenpreviously processed, the process may extract (at 550) media parameters.Such parameters may include, for instance, lyrics, key or “target key”,genre, chord progression, melodies, etc.

Next, the process may transpose (at 560) the media selection to the basekey. Such transposition may include, for instance, transferring singlenotes and/or chords to the corresponding notes and/or chords of adifferent key. For example, songs played on a keyboard may be transposedto a base key of C as this only requires use of the white keys. In somecases, the base key may depend on the level of the user. For instance, abeginner may play all tunes in the key of C on keyboard, while anintermediate or advanced player may play tunes in multiple keys. Asanother example, a singer with limited range (and/or skill) may select abase key that best fits the capabilities of the singer while an advancedsinger may utilize other base keys as appropriate.

By transposing the selection to a base key, some embodiments are able toprovide a consistent interface (e.g., a set of colors where each colorrepresents a chord or other group of notes), such that users may playmultiple tunes of any original key while using the same sets of controlelements, as designated by the matching colors. In addition, although aparticular base key may be used (e.g., C major) different notes (than C)within the key may serve as roots to allow additional variations. As oneexample, A minor shares the same notes as C major.

After transposing (at 560) the media item to the base key, the processmay link (at 570) the transposed content to the lyrics (and/or othervisual cues). The lyrics may be examined to determine chord changepoints and the color of the lyrical text be set to the appropriate chordrepresentation. Alternatively and/or conjunctively, other indicators maybe used (e.g., colored boxes, bubbles, etc.). In addition, variousreferences may be provided (e.g., a moving timeline, a ball that bouncesalong lyric text, a metronome or drum beat, etc.). Some embodiments maygenerate a reduced version of the original media content (e.g., keyboardand singing elements may be eliminated while other instruments areincluded).

The media content may be otherwise manipulated and/or synchronized tothe playback cues. Some embodiments may include video or other graphicalrepresentations of the various instrument control elements, where therepresentations are synchronized to the timeline or other progressindicator. For instance, a top view of the set of labelled keys may bepresented where the active keys are highlighted (e.g., via color change,movement, etc.) such that a player can follow the movements of the keysas associated with the displayed colors.

If the process determines (at 540) that the selected media has beenpreviously processed, the process may retrieve (at 580) the interactiveplayable media. After retrieving (at 580) or generating (at 550-570) themedia, the process may provide (at 590) the media to the user and thenmay end. The media may be provided using an interface such as GUI 400described above.

One of ordinary skill in the art will recognize that process 500 may beperformed in various appropriate ways without departing from the scopeof the disclosure. For instance, various operations may be omitted,other operations may be included, and/or the operations may be performedin a different order than shown. As another example, the process may bedivided into various sub-processes and/or included in a macro process.Some or all of the operations may be performed iteratively, asappropriate.

V. Usage Scenarios

There are many variations and use of this disclosure to inspire andsupport advanced musical skills. Because the system supports the melodyas a way of accompaniment, a user is allowed to work with and/or createa variety of styles. These styles are limited only by the imaginationand creativity of the user.

Using the colors, a user may determine how the triads are pronounced.For instance, a user may add a rhythm pattern or change the tempo at theverse or otherwise vary the performance to suit the style and taste ofthe user.

Users are able to utilize the system of some embodiments to memorizeand/or create a structure for an original tune. The system may be ableto support and create vocal riffs, unique melodies or implement othermelodies onto a progression pattern.

Some embodiments may support the use of two songs within one keystructure, commonly known as a “mash up”. Two different songs that havesimilar chord progressions and tempo may be used to create two distinctrecognizable melodies in one song. This system supports the process ofcombination and encourages user exploration of further musicality.

Some embodiments allow creation of an original tune using simplifiedtriad tools, thus enabling music creators by requiring one less task tomaster. While determining a root foundation in conjunction with amelody, a music composer is able to allow one component to remain freeand easy while strategizing the other facet of music construction. Thecolor coded decals on the keyboard encourage the creator to test andexplore different musical directions with the ease of color options.This process can inspire melodies or facilitate the natural creation ofmelodies, composition, direction, and/or chord progression.

VI. Computer System

Many of the processes and modules described above may be implemented assoftware processes that are specified as one or more sets ofinstructions recorded on a non-transitory storage medium. When theseinstructions are executed by one or more computational element(s) (e.g.,microprocessors, microcontrollers, digital signal processors (DSPs),application-specific integrated circuits (ASICs), field programmablegate arrays (FPGAs), etc.) the instructions cause the computationalelement(s) to perform actions specified in the instructions.

In some embodiments, various processes and modules described above maybe implemented completely using electronic circuitry that may includevarious sets of devices or elements (e.g., sensors, logic gates, analogto digital converters, digital to analog converters, comparators, etc.).Such circuitry may be able to perform functions and/or features that maybe associated with various software elements described throughout.

FIG. 6 illustrates a schematic block diagram of an exemplary computersystem 600 used to implement some embodiments. For example, the systemdescribed above in reference to FIG. 3 may be at least partiallyimplemented using computer system 600. As another example, the GUIdescribed in reference to FIG. 4 and the process described in referenceto FIG. 5 may be at least partially implemented using sets ofinstructions that are executed using computer system 600.

Computer system 600 may be implemented using various appropriatedevices. For instance, the computer system may be implemented using oneor more personal computers (PCs), servers, mobile devices (e.g., asmartphone), tablet devices, and/or any other appropriate devices. Thevarious devices may work alone (e.g., the computer system may beimplemented as a single PC) or in conjunction (e.g., some components ofthe computer system may be provided by a mobile device while othercomponents are provided by a tablet device).

As shown, computer system 600 may include at least one communication bus605, one or more processors 610, a system memory 615, a read-only memory(ROM) 620, permanent storage devices 625, input devices 630, outputdevices 635, various other components 640 (e.g., a graphics processingunit), and one or more network interfaces 645.

Bus 605 represents all communication pathways among the elements ofcomputer system 600. Such pathways may include wired, wireless, optical,and/or other appropriate communication pathways. For example, inputdevices 630 and/or output devices 635 may be coupled to the system 600using a wireless connection protocol or system.

The processor 610 may, in order to execute the processes of someembodiments, retrieve instructions to execute and/or data to processfrom components such as system memory 615, ROM 620, and permanentstorage device 625. Such instructions and data may be passed over bus605.

System memory 615 may be a volatile read-and-write memory, such as arandom access memory (RAM). The system memory may store some of theinstructions and data that the processor uses at runtime. The sets ofinstructions and/or data used to implement some embodiments may bestored in the system memory 615, the permanent storage device 625,and/or the read-only memory 620. ROM 620 may store static data andinstructions that may be used by processor 610 and/or other elements ofthe computer system.

Permanent storage device 625 may be a read-and-write memory device. Thepermanent storage device may be a non-volatile memory unit that storesinstructions and data even when computer system 600 is off or unpowered.Computer system 600 may use a removable storage device and/or a remotestorage device as the permanent storage device.

Input devices 630 may enable a user to communicate information to thecomputer system and/or manipulate various operations of the system. Theinput devices may include keyboards, cursor control devices, audio inputdevices and/or video input devices. Output devices 635 may includeprinters, displays, and/or audio devices. Some or all of the inputand/or output devices may be wirelessly or optically connected to thecomputer system.

Other components 640 may perform various other functions. Thesefunctions may include performing specific functions (e.g., graphicsprocessing, sound processing, etc.), providing storage, interfacing withexternal systems or components, etc.

Finally, as shown in FIG. 6, computer system 600 may be coupled to oneor more networks 650 through one or more network interfaces 645. Forexample, computer system 600 may be coupled to a web server on theInternet such that a web browser executing on computer system 600 mayinteract with the web server as a user interacts with an interface thatoperates in the web browser. Computer system 600 may be able to accessone or more remote storages 660 and one or more external components 665through the network interface 645 and network 650. The networkinterface(s) 645 may include one or more application programminginterfaces (APIs) that may allow the computer system 600 to accessremote systems and/or storages and also may allow remote systems and/orstorages to access computer system 600 (or elements thereof).

As used in this specification and any claims of this application, theterms “computer”, “server”, “processor”, and “memory” all refer toelectronic devices. These terms exclude people or groups of people. Asused in this specification and any claims of this application, the term“non-transitory storage medium” is entirely restricted to tangible,physical objects that store information in a form that is readable byelectronic devices. These terms exclude any wireless or other ephemeralsignals.

It should be recognized by one of ordinary skill in the art that any orall of the components of computer system 600 may be used in conjunctionwith some embodiments. Moreover, one of ordinary skill in the art willappreciate that many other system configurations may also be used inconjunction with some embodiments or components of some embodiments.

In addition, while the examples shown may illustrate many individualmodules as separate elements, one of ordinary skill in the art wouldrecognize that these modules may be combined into a single functionalblock or element. One of ordinary skill in the art would also recognizethat a single module may be divided into multiple modules.

The foregoing relates to illustrative details of exemplary embodimentsand modifications may be made without departing from the spirit andscope of the disclosure as defined by the following claims.

I claim:
 1. A method that provides interactive music playback media fora keyboard, the method comprising: establishing a base key, wherein eachparticular bass key label from a set of bass key labels is affixed toeach diatonic key along a first octave of the keyboard starting from atonic of the bass key and each particular bass key label includes aparticular unique color from a set of colors, and wherein eachparticular harmony key label from a set of harmony key labels is affixedto each diatonic key along a second octave of the keyboard starting fromthe tonic of the base key and each particular harmony key label includesa plurality of colors from the set of colors; receiving, via a graphicaluser interface of a user device, a selection of a media item; extractingmusical parameters associated with the media item, the musicalparameters including a target key, a chord progression, and a set oflyrics; transposing the media item from the target key to the base key;linking the transposed media item to at least one musical parameterassociated with the media item; generating a visual representation ofthe transposed media item including a set of visual indicators, eachvisual indicator including at least one particular color from the set ofcolors and an element from the set of lyrics; and displaying the visualrepresentation at a display of the user device, wherein establishing thebase key, extracting the musical parameters, transposing the media item,linking the transposed media item, and generating the visualrepresentation are performed by a processor of at least one of the userdevice and a server.
 2. The method of claim 1, wherein each unique coloris associated with a plurality of harmony key labels.
 3. The method ofclaim 1, wherein the base key is associated with a particular targetinstrument.